This invention relates to data processing for three-dimensional displays.
Image based rendering can be used to generate data for realistic representation of three-dimensional (3-D) images on 3-D displays, such as multi-view or holographic 3-D displays. Images of an object (or a scene) captured from several viewpoints are used for generating new images of the object as viewed from different viewpoints. In a multi-view 3-D display, the 3-D image of the object can be viewed from multiple view angles. A 3-D display creates imagery of the object that provides one or more stereoscopic depth cues, such as motion parallax, to create the perception of a 3-D scene. The 3-D display processes the light field of the 3-D scene and computes trajectories of the constituent light rays that exit an image surface to generate the light field.
The term “light field” herein refers to a collection of light rays that emanate from particular positions and propagate along particular paths. Some of the light rays may be occluded and not seen from certain angles. The phrase “light field of an object” refers to the collection of light rays that emanate from the object. The phrase “light field rendering” refers to generating data that can be used by a 3-D display to project light rays from appropriate positions along appropriate directions to generate the light field.
As an example, referring to FIG. 1, to generate an image point 2 in 3-D space, several light rays (e.g., 4a, 4b, 4c, and 4d) are projected from points 14a to 14d positioned on a horizontal region 6 of an image surface 8. The rays converge at and emanate from the image point 2, and each light ray has a color and brightness that represent the color and brightness of the image point 2 when viewed along the direction of the light ray. An observer at a first position 10 receives light rays 4a and 4b in his left and right eyes. The slight difference in the rays 4a and 4b provides a depth cue. When the observer moves horizontally to a second position 12, he sees the rays 4c and 4d, whose difference also generates a depth cue. As the observer moves, he has the illusion of viewing the same image point 2 from different viewing angles even though the light rays are actually projected from different locations on the image surface 8. In this example, the light field for the image point 2 includes the light rays 4a to 4d, and may include additional light rays. Rendering the light field includes determining the projection locations 14a to 14d, the projection directions of the light rays 4a to 4d, and the color and brightness of the light rays 4a to 4d. 
Referring to FIG. 2, a multi-view 3-D display 24 includes spatial light modulators (SLM) 20, such as DIGITAL MICROMIRROR DEVICES™ (DMDs) from Texas Instruments, Inc., Plano, Tex. Each DMD has an array of rows and columns of pixels that can be individually switched on and off, and can be programmed with a particular pattern to modulate light from a light source (not shown) to form presentation images that are projected by projection optics (not shown) onto the image surface 22. Projecting a presentation image means projecting light rays that represent the presentation image. The collection of presentation images (or the collection of light rays representing the presentation images) generates a light field that approximates the light field an observer would see from a real 3-D object.
In one example, an array of SLMs are positioned along a horizontal direction, and the image surface 22 is a screen that diffuses light rays vertically but does not change the horizontal propagation directions of the light rays. This differs from a normal screen, such as the screen of a conventional television or computer monitor, in which each point on the screen produces light rays that propagate essentially as a hemispherical wavefront (in various vertical and horizontal directions), so that the same point can be seen from different vertical and horizontal view angles. The image surface 22 does not change the horizontal propagation direction of a light ray, so that unless the observer is located at a particular horizontal position, he will not see the light ray.
For each point on the 3-D object, different light rays representing the point as viewed from different directions are projected along different horizontal directions, so that the observer sees different light rays from different horizontal view angles. This allows the display 24 to generate 3-D images with horizontal parallax, in which different views of the object can be perceived when the observer moves horizontally (e.g., from position 10 to 12), but substantially the same view of the object is perceived when the observer moves vertically.
Alternatively, a two-dimensional array of SLMs and an image surface 22 that does not diffuse light rays can be used to generate a 3-D image with full parallax, in which different views of the object can be perceived when the observer moves either horizontally or vertically.
Rather than using multiple spatial light modulators, one spatial light modulator (or a subset of the array of SLMs) can be used. In this case, the presentation images are synthesized in a time-division-multiplexed manner so that different presentation images are projected from different positions at different times. At any instant, a subset of the required rays, with various angles, emanate through the image surface 24. The collection of the light rays integrated over a short period of time approximates complete wave fronts emanating from the 3-D object. If the SLM operates sufficiently fast, and if the projection optics project the different presentation images from different locations within a sufficiently short period of time (such as 1/30 second), persistence of vision causes the image of the 3-D object to look complete. As a result, the light rays look to an observer as if they had all been emitted from the 3-D object at the same time.
An example of a multi-view 3-D display is disclosed in Provisional Patent Application 60/555,602, “Scanned Multiview Three-Dimensional Display,” filed Mar. 23, 2004, herein incorporated by reference.